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ABSTRACT 

The paper examines varieties of program assessment- m 
institutions of higher education and proposes a procedure to 
implement systematic assessment of program outccmes. Presented in a 
qu€stion-and-answer format, the paper first addresses common faculty 
concerns about formal assessment m sections titled, "Why Not Use GPA 
(oi.'au»? Point Average) and "Isn't Placement Our Best Indicator'^" 
Assessment development and implementation are adaressed m answers to 
the following questions: How much will this cost? How do we get 
started'^ Will standardized exams assess what we teach? Will local 
exams be any good'^ Will students cooperate m assessment? Will my 
work be recognized'^ How much time will this take? and What good will 
assessment do? The paper streses the importance of: viewing 
assessment as a priority ^or everyone within the institution; 
building assessment into established procedures (such as curriculurr 
review, strategic planning, and peer review); and rewarding faculty 
and students for their involvement in assessment. Includes seven 
references, (DB) 
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QUESTIONS FACULTY ASK ABOUT ASSESSMENT 
Trudy W. Banta 



"Assessment is the means by which students learn to lead an examined 
life, and colleges cannot exemplify that goal for students if they do not 
adequately evaluate their own programs of teaching" [Ehrmann, 1988]. This 
rationale, coupled with the societal pressures of consumerism and cost 
containment and recent critiques of higher education from within its own 
ranks [Association of American Colleges, 1985; Boyer, 1987], has produced 
unprecedented interest and activity in the area of assessing the outcomes 
of higher education. 

The National Governors' Association has reported that 24 states now 
require publicly- supported colleges and universities to engage in 
assessment, and 12 other states are considering such a requirement 
[National Governors' Association, 1988]. All six regional accrediting 
associations include outcomes assessment in their standards for 
accreditation of post-secondary institutions. Not surpri.singly , the 

American Council on Education's Campus Trends . ISM revealed that 

assessment activities were underway at two-thirds of the public 
institutions and 40 percent of the independent colleges surveyed for this 
annual study [El-Khawas, 1988]. 

The following remarks will be based somewhat on my experience in 
visiting almost 40 campuses in 21 states over the last four years, but 
they are shaped primarily by my work as coordinator of assessment at the 
University of Tennessee, Knoxville, a position I have held since 1982. 



"Why not use CPA?" 

A common initial reaction of faculty to the idea of assessing student 
outcomes in a comprehensive way, that is, above and beyond the traditional 
classroom examination, is to ask, "How does this differ from grade point 
average? Iza' t cumulative grade point average the best indication of what 
students have learned in college?" 

Based on most studies of grade point average (for example, Milton, 
Pollio, and Eison, 1986) the answer to tuis question i-« a resounding "No." 
Cumulativi^ grade point average is a product of ability, motivation, and 
just plain luck. It is most closely related to student ability, and bears 
little relationship to success in life beyond college. 

The whole of the educational experience is much greater than the sum 
of a series of individual courses. Without some comprehensive product- - 
preferably a series of products- -to show how well a student can integrate 
what he or she has learned, students may proceed through the curriculum 
and graduate without leaving the faculty with any sense of how they will 
respond when confronted with the larger problems they will encounter in 
the world beyond the classroom. 

"Isn't placement our best indicator?" 

At research universities, faculty are wont to say, "We judge our 
success by the placement of our students in the best graduate programs." 

erJc , 



At 2-year institutions and in selected disciplines at 4-year colleges, 
faculty contend that the best indicator of their success with students is 
a substantial record of job placement in the fields for which students 
have been prepared. These faculty ask. "Why should we add another measure 
of program outcomes when we are satisfied with the information we get from 
placement data?" 

Information about student placement is an important indicator of 
program success, and should be used in a profile of indicators of 
effectiveness. This information is not sufficient in and of itself 
however, because in most postsecondary institutions across the country, 
graduate education is not a goal of a majority of the students. 
Therefore, it is not appropriate to base the success of the entire program 
on the success of a minority of the student population. Job placement is 
not a goal of every student, either. Moreover, employment in a field that 
appears to be unrelated to a student's primary discipline may not in the 
long run prove to be a negative consequence. 

In any case, following the career success of former students is time- 
consuming, difficult, and expensive. When we ask faculty "How many of 
your .students have you tracked ..n graduate school or in jobs?" we often 
find that they have no systematic means of doing this work. They rely 
simply on information received from those students who voluntarily keep 
in touch with their former professors. They rarely write down the 
comments they do receive, and when asked to recall students' reactions, 
selective memory may create a more positive summary than the facts 
warrant. 
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"How much will this cost?" 



After some of the initial defensiveness is alleviated and faculty begin 
to entertain the possibility that they will become involved in assessment, 
a common question is "How much will assessment cost?" Most of the 
assessment offices established in the last eight years are very small- -no 
more than a couple of positions. So-e are headed by a faculty member who 
spends 25-50 percent of his or her time coordinating assessment. 

At the University of Tennessee, Knoxville, where assessment is a very 
important activity, we draw upon services in every division of the 
institution to assist in testing students, recording the results of their 
performance, and compiling reports for various audiences. On all of these 
functions, we annually spend .about ten dollars per enrolled student. This 
is an amount equal to about one tenth of one percent of the University's 
total budget. Those who have found the results of assessment activities 
to be helpful in improving the student experience consider these dollars 
well spent. 



"How do we get started?" 



Once faculty have made a commitment 
they ask, "How do we get started?" I 
responding to the question, "What do 
students? 



to do something about assessment , 
would suggest that they begin by 
we need to know to he Ip us he Ip 
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Are students learning? How do you know? How could they learn more? 
These answers will differ for every department. A good way to begin is 
with a statement of goals for student development. What is the content 
of the discipline and what are the ways of knowing in the field? (Both 
content and process should be assessed.) What are essential student 
services? What are they designed to do; what will they promote? 

"Will standardized exams assess what we teach?" 

When faculty and student services staff have thought through thair 
goals for student development, they can begin to devise ways of measuring 
student progress toward those goals. Most faculty begin by looking at 
standardized exams that may be available, and the most important question 
in connection with this review is, "If we use standardized exams, will 
they reflect what we teach?" 

The answer to this question will vary from campus to campus because 
each faculty has different strengths and, not surprisingly, differing 
goals for student development. Since standardized exams offer the 
benefits of well constructed items and norms for score comparison, a 
faculty may decide to use a standardized exam even though the test does 
not reflect all of the important areas that appear in their goals 
statement. Ideally, they will supplement the iiformaticn gained from the 
standardized instrument by developing their own measures of student 
performance. At UTK, for instance, the Franch faculty decided to use the 
National Teacher Exam Specialty Area in French for its seniors, but 
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designed their own measures of listening, speaking, reading, and writing 
in order to provide broader coverage of their curricular goals. 



"Will local exams be any good?" 



Faculty examination ol measurement instruments for use in assessment 
is often accompanied by the question. "If we develop our own measures, 
will they be any good?" Faculty themselves are the best judges of the 
goodness of a measure. If they believe they have developed a good exam 
and trur.t the results that it yields, they will use those results to 
suggest improvements in what they are doing with and for students, and 
thet is the ultimate objective of assessment. 

We can provide some advice for faculty who embark on the task of 
developing their own measures. First, use a variety of measures, not just 
one. Be especially wary of falling into the trap of devising a single 
multiple-choice exam simply because you think that is the only kind of 
student performance that can be "objectively" evaluated. Ways are 
available to make a variety of types of faculty judgment reliable, thus 
compr.rable over time. 

The items on any instrument should be clear, both to colleagues in the 
discipline and the students who will be responding to them. Both groups 
should be asked to review items before they are used for assessment. 
Respected colleagues outside the institution can be asked to compare items 
on an assessment instrument with the faculty's goals statement to 



establish the degree of congruence between the two. Then students, 

Including graduate students In the department and even students In other 

disciplines, can be asked to pilot-test the Items and offer their own 
reactions and constructive criticism. 

Aufessment Instruments should Include measures of performance that 
permit students to exhibit knowledge and skills faculty believe they will 
need for effective participation In the discipline following graduation. 
Such measures might Include portfolios of writing, artistic productions, 
videotapes of teaching and other oral presentations, poster presentations 
In the sciences, or simulations of on-the-job performance. Ovp : time 
faculty can evaluate these Instruments by following up program graduates 
and linking their future successes with performance on the comprehensive 
assessment In the major. 

"Will students cooperate In assessment?" 

As faculty become Immersed In assessment design, they begin to worry 
about student participation. A common question Is "How can we motivate 
students to take part In assessment activities?" 

The answer Is to Involve students from the beginning In the assessment 
process. Student representatives can serve on the faculty committee that 
determines the direction of assessment. Some students can assist In 
evaluating Instruments. For Instance, students on one campus may serve 
as the pllot-test subjects for Instruments designed for use on another 
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campus. Graduate students may provide feedback on strategies that will 
be used for undergraduate assessment in a department. 

The most important thing for faculty to do in gaining student 
cooperation is to communicate to students that assessment is important to 
the future of their program, that faculty are invested in the process and 
Will use the results in making improvements, and that ultimately students 
will be the beneficiaries. Faculty should present each student with their 
judgments about the student's performance. This conversation will be 
helpful to the student in making sense of the scor.s and may lead to 
further understanding of program strengths and weaknesses on the part of 
the faculty member(s) involved. 

Students should understand that their performance on assessment 
activities will give them a reading on the level of cheir achievement of 
faculty-developed program objectives, and they can use this data in 
presenting a fuller picture cf their abilities and skills to an employer. 
Having the opportunity to use assessment results in employment resumes has 
been demonstrated to be a potent factor In fostering student motivation 
to participate honestly and conscientiously. 

"Will my work be recognized?" 

Faculty will almost certainly ask. "Uill I be compensated or recognized 
for my work in assessment?" Many program administrators with assessment 
experience view faculty involvement as essential to effective performance 



id ( * 

ERIC 



9 

in their departments. They have negotiated agreements with faculty that 
provide released time or extra compensation for assessment design an-^ 
administration. In addition, the institution may provide special awards 
and recognition for faculty who make outstanding contributions to 
assessment . 

"How much time will this take?" 

"Will this take too much time from my preparation for teaching or from 
my research?" Assessment certainly can be used to improve classroom 
teaching. Joint afforts to develop goals and specific objectives for the 
curriculum make each faculty member more cognizant of related objectives 
for their own courses. They then begin to share these objectives with 
students, and the result is a more systematic approach throughout the 
department to the achievement of specified goals. Heretofore, the 
adherence to specific goals for a discipline, and especially for general 
education, has been sporadic throughout the institution. Activities 
associated with preparing for assessment have promoted a sense of shared 
purpose among faculty. The use of rational planning is promoted by an 
emphasis cn assessment. 

At UTK faculty \ uwe been encouraged through a series of awards provided 
by the provost to make assessment a subject of their research. During 
1988-89 faculty in four colleges received institutional funding for work 
on projects related to improving their departmental assessment 
instruments. Faculty throughout the institution have contributed to 
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assessn>ent Initiatives developing within their own disciplinary and 
professional organizations. A bibliography of assessment-related 
presentations and publications prepared by UTK faculty now contains over 
100 entries. 



"What good will assessment do?" 

A final and Important quesclon is. "What good will assessment do?" Tom 
Moran ac the State University of New York at Plattsburgh. says that 
assessment on his campus has Initiated a dialogue about learning that has 
led faculty to become more involved in scholarly activity related to their 
teaching. Assessment has stimulated collaboration, among faculty, between 
faculty and students, and among student! as ^hey talk about assessment 
procedures and results. As a consequence of this coming together, faculty 
and students increasingly -^iew learning as a shared responsibility. 

On a number of campuses - King's College in Pennsylvania is a good 
example assessment activities have provided a means of rejuvenating 
faculty experiencing burnout. Assessment is an endeavor that calls for 
innovative thinking and new approaches to curriculum organization, 
teaching, and classroom testing. Faculty are stimulated to find new ways 
to involve students in learning, in classroom interaction and assignments 
and on course exams . 

Perhaps the most Important benefit of all is that assessment can 
enhance student learning. Faculty approach teaching and test-making more 
systematically and provide increasing amounts of feedback to students 
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concerning their performances. Students are learning more about their 
strengths and weaknesses and about alternative paths to improvement. 



Some Recommendations 



Having considered some of the questions faculty ask about assessment, 
it may be in order to suggert a few general recommendations. First, 
assessment must be viewed as a priority for everyone within the 
institution, from the top administrators to faculty to students. All of 
these groups should be involv 1 in discussion about the purposes and uses 
of assessment. Administrators must establish an atmosphere of trust in 
which facult> feel free to discover curricular and instructional 
weaknesses and to ask for assistance in correcting thase weaknesses. 

Assessment should be built into established procedures that have 
meaning in the life of the institution, such as curriculum review, 
strategic planning, and peer review. This gives assessment a xaU o\ ) 
d'etre that will assure its continuation. 

Faculty should be recognized and rewarded for their involvement in 
assessment and students should be recognized for their participation and 
conscientious effort in assessment activities. It is very important to 
stress faculty and student interaction during the design and 
administration of assessment techniques and especially in the analysis of 
results. Students can gain information about their own development, and 
faculty can gain insight into the success of their assessment techniques 
by talking with students about their reactions to assessment procedures. 
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